Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations.

Background: Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here, we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022. Methods: We used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported by a standardised source for 32 countries over the next 1-4 weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models' predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models' forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models' past predictive performance. Results: Over 52 weeks, we collected forecasts from 48 unique models. We evaluated 29 models' forecast scores in comparison to the ensemble model. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 83% of participating models' forecasts of incident cases (with a total N=886 predictions from 23 unique models), and 91% of participating models' forecasts of deaths (N=763 predictions from 20 models). Across a 1-4 week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over 4 weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models. Conclusions: Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than 2 weeks. Funding: AA, BH, BL, LWa, MMa, PP, SV funded by National Institutes of Health (NIH) Grant 1R01GM109718, NSF BIG DATA Grant IIS-1633028, NSF Grant No.: OAC-1916805, NSF Expeditions in Computing Grant CCF-1918656, CCF-1917819, NSF RAPID CNS-2028004, NSF RAPID OAC-2027541, US Centers for Disease Control and Prevention 75D30119C05935, a grant from Google, University of Virginia Strategic Investment Fund award number SIF160, Defense Threat Reduction Agency (DTRA) under Contract No. HDTRA1-19-D-0007, and respectively Virginia Dept of Health Grant VDH-21-501-0141, VDH-21-501-0143, VDH-21-501-0147, VDH-21-501-0145, VDH-21-501-0146, VDH-21-501-0142, VDH-21-501-0148. AF, AMa, GL funded by SMIGE - Modelli statistici inferenziali per governare l'epidemia, FISR 2020-Covid-19 I Fase, FISR2020IP-00156, Codice Progetto: PRJ-0695. AM, BK, FD, FR, JK, JN, JZ, KN, MG, MR, MS, RB funded by Ministry of Science and Higher Education of Poland with grant 28/WFSN/2021 to the University of Warsaw. BRe, CPe, JLAz funded by Ministerio de Sanidad/ISCIII. BT, PG funded by PERISCOPE European H2020 project, contract number 101016233. CP, DL, EA, MC, SA funded by European Commission - Directorate-General for Communications Networks, Content and Technology through the contract LC-01485746, and Ministerio de Ciencia, Innovacion y Universidades and FEDER, with the project PGC2018-095456-B-I00. DE., MGu funded by Spanish Ministry of Health / REACT-UE (FEDER). DO, GF, IMi, LC funded by Laboratory Directed Research and Development program of Los Alamos National Laboratory (LANL) under project number 20200700ER. DS, ELR, GG, NGR, NW, YW funded by National Institutes of General Medical Sciences (R35GM119582; the content is solely the responsibility of the authors and does not necessarily represent the official views of NIGMS or the National Institutes of Health). FB, FP funded by InPresa, Lombardy Region, Italy. HG, KS funded by European Centre for Disease Prevention and Control. IV funded by Agencia de Qualitat i Avaluacio Sanitaries de Catalunya (AQuAS) through contract 2021-021OE. JDe, SMo, VP funded by Netzwerk Universitatsmedizin (NUM) project egePan (01KX2021). JPB, SH, TH funded by Federal Ministry of Education and Research (BMBF; grant 05M18SIA). KH, MSc, YKh funded by Project SaxoCOV, funded by the German Free State of Saxony. Presentation of data, model results and simulations also funded by the NFDI4Health Task Force COVID-19 (https://www.nfdi4health.de/task-force-covid-19-2) within the framework of a DFG-project (LO-342/17-1). LP, VE funded by Mathematical and Statistical modelling project (MUNI/A/1615/2020), Online platform for real-time monitoring, analysis and management of epidemic situations (MUNI/11/02202001/2020); VE also supported by RECETOX research infrastructure (Ministry of Education, Youth and Sports of the Czech Republic: LM2018121), the CETOCOEN EXCELLENCE (CZ.02.1.01/0.0/0.0/17-043/0009632), RECETOX RI project (CZ.02.1.01/0.0/0.0/16-013/0001761). NIB funded by Health Protection Research Unit (grant code NIHR200908). SAb, SF funded by Wellcome Trust (210758/Z/18/Z).

On the contact tracing for COVID-19: A simulation study☆

Background: Contact tracing is one of the most effective non-pharmaceutical interventions in the COVID-19 pandemic. This study uses a multi-agent model to investigate the impact of four types of contact tracing strategies to prevent the spread of COVID-19. Methods: In order to analyse individual contact tracing in a reasonably realistic setup, we construct an agent-based model of a small municipality with about 60.000 inhabitants (nodes) and about 2.8 million social contacts (edges) in 30 different layers. Those layers reflect demographic, geographic, sociological and other patterns of the TTWA (Travel-to-work-area) Hodonín in Czechia. Various data sources such as census, land register, transport data or data reflecting the shopping behaviour, were employed to meet this purpose. On this multi-graph structure we run a modified SEIR model of the COVID-19 dynamics. The parameters of the model are calibrated on data from the outbreak in the Czech Republic in the period March to June 2020. The simplest type of contact tracing follows just the family, the second tracing version tracks the family and all the work contacts, the third type finds all contacts with the family, work contacts and friends (leisure activities). The last one is a complete (digital) tracing capable of recalling any and all contacts. We evaluate the performance of these contact tracing strategies in four different environments. First, we consider an environment without any contact restrictions (benchmark);second with strict contact restriction (replicating the stringent non-pharmaceutical interventions employed in Czechia in the spring 2020);third environment, where the measures were substantially relaxed, and, finally an environment with weak contact restrictions and superspreader events (replicating the situation in Czechia in the summer 2020). Findings: There are four main findings in our paper. 1. In general, local closures are more effective than any type of tracing. 2. In an environment with strict contact restrictions there are only small differences among the four contact tracing strategies. 3. In an environment with relaxed contact restrictions the effectiveness of the tracing strategies differs substantially. 4. In the presence of superspreader events only complete contact tracing can stop the epidemic. Interpretation: In situations, where many other non-pharmaceutical interventions are in place, the specific extent of contact tracing may not have a large influence on their effectiveness. In a more relaxed setting with few contact restrictions and larger events the effectiveness of contact tracing depends heavily on their extent.

On the contact tracing for COVID-19: A simulation study.

BACKGROUND: Contact tracing is one of the most effective non-pharmaceutical interventions in the COVID-19 pandemic. This study uses a multi-agent model to investigate the impact of four types of contact tracing strategies to prevent the spread of COVID-19. METHODS: In order to analyse individual contact tracing in a reasonably realistic setup, we construct an agent-based model of a small municipality with about 60.000 inhabitants (nodes) and about 2.8 million social contacts (edges) in 30 different layers. Those layers reflect demographic, geographic, sociological and other patterns of the TTWA (Travel-to-work-area) Hodonín in Czechia. Various data sources such as census, land register, transport data or data reflecting the shopping behaviour, were employed to meet this purpose. On this multi-graph structure we run a modified SEIR model of the COVID-19 dynamics. The parameters of the model are calibrated on data from the outbreak in the Czech Republic in the period March to June 2020. The simplest type of contact tracing follows just the family, the second tracing version tracks the family and all the work contacts, the third type finds all contacts with the family, work contacts and friends (leisure activities). The last one is a complete (digital) tracing capable of recalling any and all contacts. We evaluate the performance of these contact tracing strategies in four different environments. First, we consider an environment without any contact restrictions (benchmark); second with strict contact restriction (replicating the stringent non-pharmaceutical interventions employed in Czechia in the spring 2020); third environment, where the measures were substantially relaxed, and, finally an environment with weak contact restrictions and superspreader events (replicating the situation in Czechia in the summer 2020). FINDINGS: There are four main findings in our paper. 1. In general, local closures are more effective than any type of tracing. 2. In an environment with strict contact restrictions there are only small differences among the four contact tracing strategies. 3. In an environment with relaxed contact restrictions the effectiveness of the tracing strategies differs substantially. 4. In the presence of superspreader events only complete contact tracing can stop the epidemic. INTERPRETATION: In situations, where many other non-pharmaceutical interventions are in place, the specific extent of contact tracing may not have a large influence on their effectiveness. In a more relaxed setting with few contact restrictions and larger events the effectiveness of contact tracing depends heavily on their extent.

Protection provided by vaccination, booster doses and previous infection against covid-19 infection, hospitalisation or death over time in Czechia.

Studies demonstrating the waning of post-vaccination and post-infection immunity against covid-19 generally analyzed a limited range of vaccines or subsets of populations. Using Czech national health data from the beginning of the covid-19 pandemic till November 20, 2021 we estimated the risks of reinfection, breakthrough infection, hospitalization and death by a Cox regression adjusted for sex, age, vaccine type and vaccination status. Vaccine effectiveness against infection declined from 87% at 0-2 months after the second dose to 53% at 7-8 months for BNT162b2 vaccine, from 90% at 0-2 months to 65% at 7-8 months for mRNA-1273, and from 83% at 0-2 months to 55% at 5-6 months for the ChAdOx1-S. Effectiveness against hospitalization and deaths declined by about 15% and 10%, respectively, during the first 6-8 months. Boosters (third dose) returned the protection to the levels observed shortly after dose 2. In unvaccinated, previously infected individuals the protection against infection declined from 97% after 2 months to 72% at 18 months. Our results confirm the waning of vaccination-induced immunity against infection and a smaller decline in the protection against hospitalization and death. Boosting restores the original vaccine effectiveness. Post-infection immunity also decreases over time.

Delays, Masks, the Elderly, and Schools: First Covid-19 Wave in the Czech Republic.

Running across the globe for nearly 2 years, the Covid-19 pandemic keeps demonstrating its strength. Despite a lot of understanding, uncertainty regarding the efficiency of interventions still persists. We developed an age-structured epidemic model parameterized with epidemiological and sociological data for the first Covid-19 wave in the Czech Republic and found that (1) starting the spring 2020 lockdown 4 days earlier might prevent half of the confirmed cases by the end of lockdown period, (2) personal protective measures such as face masks appear more effective than just a realized reduction in social contacts, (3) the strategy of sheltering just the elderly is not at all effective, and (4) leaving schools open is a risky strategy. Despite vaccination programs, evidence-based choice and timing of non-pharmaceutical interventions remains an effective weapon against the Covid-19 pandemic.

Protection by Vaccines and Previous Infection Against the Omicron Variant of Severe Acute Respiratory Syndrome Coronavirus 2.

BACKGROUND: The Omicron variant of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) evades immunity conferred by vaccines and previous infections. METHODS: We used a Cox proportional hazards model and a logistic regression on individual-level population-wide data from the Czech Republic to estimate risks of infection and hospitalization, including severe states. RESULTS: A recent (≤2 months) full vaccination reached vaccine effectiveness (VE) of 43% (95% confidence interval [CI], 42%-44%) against infection by Omicron compared to 73% (95% CI, 72%-74%) against Delta. A recent booster increased VE to 56% (95% CI, 55%-56%) against Omicron infection compared to 90% (95% CI, 90%-91%) for Delta. The VE against Omicron hospitalization of a recent full vaccination was 45% (95% 95% CI, 29%-57%), with a recent booster 87% (95% CI, 84%-88%). The VE against the need for oxygen therapy due to Omicron was 57% (95% CI, 32%-72%) for recent vaccination, 90% (95% CI, 87%-92%) for a recent booster. Postinfection protection against Omicron hospitalization declined from 68% (95% CI, 68%-69%) at ≤6 months to 13% (95% CI, 11%-14%) at >6 months after a previous infection. The odds ratios for Omicron relative to Delta were 0.36 (95% CI, .34-.38) for hospitalization, 0.24 (95% CI, .22-.26) for oxygen, and 0.24 (95% CI, .21-.28) for intensive care unit admission. CONCLUSIONS: Recent vaccination still brings substantial protection against severe outcome for Omicron.

Importance of vaccine action and availability and epidemic severity for delaying the second vaccine dose.

Following initial optimism regarding potentially rapid vaccination, delays and shortages in vaccine supplies occurred in many countries during spring 2021. Various strategies to counter this gloomy reality and speed up vaccination have been set forth, of which the most popular has been to delay the second vaccine dose for a longer period than originally recommended by the manufacturers. Controversy has surrounded this strategy, and overly simplistic models have been developed to shed light on this issue. Here we use three different epidemic models, all accounting for then actual COVID-19 epidemic in the Czech Republic, including the real vaccination rollout, to explore when delaying the second vaccine dose by another 3 weeks from 21 to 42 days is advantageous. Using COVID-19-related deaths as a quantity to compare various model scenarios, we find that the way of vaccine action at the beginning of the infection course (preventing infection and symptoms appearance), mild epidemic and sufficient vaccine supply rate call for the original inter-dose period of 21 days regardless of vaccine efficacy. On the contrary, for the vaccine action at the end of infection course (preventing severe symptoms and death), severe epidemic and low vaccine supply rate, the 42-day inter-dose period is preferable, at any plausible vaccine efficacy.